Media channel identification and action with multi-match detection based on reference stream comparison

ABSTRACT

A computing system will compare various reference fingerprints each representing a reference media stream broadcast on a different respective known channel, and the computing system will determine that a plurality of the reference fingerprints match each other, thus defining a multi-match group of the matching reference fingerprints. Further, the computing system will determine that a query fingerprint representing a media stream being rendered by a media presentation device matches the multi-match group, thus raising a question of which channel is carrying the media stream that is being rendered by the media presentation device. By considering one or more attributes of the query fingerprint, the server may then disambiguate and thereby determine the channel at issue, and the server may in turn take channel-specific action.

REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/246,746, filed Jan. 14, 2019, which is a continuation of U.S. patentapplication Ser. No. 16/015,811, filed Jun. 22, 2018, which issued asU.S. Pat. No. 10,225,605 and is a continuation of U.S. patentapplication Ser. No. 15/443,580, filed Feb. 27, 2017, which issued asU.S. Pat. No. 10,104,426 and which claims priority to U.S. ProvisionalPatent Application No. 62/301,616, filed Feb. 29, 2016, the entirety ofeach of which is hereby incorporated by reference. In addition, thedisclosures of the following applications are hereby incorporated hereinby reference in their entirety: (i) U.S. Patent Application Pub. No.2015/0181263, (ii) U.S. patent application Ser. No. 15/179,143, filedJun. 10, 2016, (iii) U.S. patent application Ser. No. 15/222,405, filedJul. 28, 2016, (iv) U.S. patent application Ser. No. 15/253,354, filedAug. 31, 2016, and (v) U.S. patent application Ser. No. 15/343,895,filed Nov. 4, 2016.

BACKGROUND

A typical media presentation device operates to receive an analog ordigital media stream representing media content such as video and/oraudio content and to render and present the media content on a userinterface such as a display screen and/or an audio speaker. Examples ofsuch devices include, without limitation, televisions, computermonitors, projection systems, loudspeakers, headphones, and the like.

In many cases, such a media presentation device may be in communicationwith a receiver, such as a local set top box or other similar device ora remote server, that has access to numerous discrete channels of mediacontent and that can selectively deliver a given such channel's mediacontent to the media presentation device for playout.

By way of example, a television may be communicatively linked with acable-TV set top box that has access to a set of cable-TV channels, andthe set top box may be configured to receive user input selecting aparticular channel and to responsively tune to the selected channel andoutput to the television the media content of the selected channel, forpresentation of the media content to a user. As another example, aloudspeaker may be communicatively linked with a radio that has accessto a set of radio stations, and the radio may be configured to receiveuser input selecting a particular channel (station) and to responsivelytune to the selected channel and output to the loudspeaker the mediacontent of the selected channel, for presentation of the media contentto a user. And as still another example, a media presentation devicesuch as a display or speaker may be communicatively linked with acomputer that has access to a myriad of online streaming media channels,and the computer may be configured to receive user input selecting aparticular channel and to responsively initiate receipt and output ofthe selected media content and provide the media content to the displayand/or speaker for presentation to a user.

SUMMARY

When a media presentation device receives and renders media content, themedia presentation device may not have an indication of which channelcarried the media content. A receiver or other device that selectivelytunes to the channel and receives the media content and that providesthe media content to the media presentation device may have suchinformation, but the media presentation device that receives the mediacontent from that other device may not have such information. Forexample, if a television is coupled with a cable-TV set top box and auser selects a particular cable channel on the set top box, the set topbox may thereby have an indication of the selected channel as thechannel on which the set top box is then receiving a media content thatis being output to the television. But the television itself may merelyreceive and render the media content and may have no indication of theselected channel.

For various reasons, however, it may be useful to determine which ofvarious channels is the channel that carries the content being renderedby a media presentation device. Further, it may be useful to do sowithout receiving from a channel-selection device (e.g., receiver orremote control) a report of the channel to which that device is tuned,and perhaps without any involvement of the channel-selection device. Forinstance, it may be useful for the media presentation device itself,and/or a network server working in cooperation with the mediapresentation device, to determine the channel based on an evaluation ofthe media content that is being rendered by the media presentationdevice. Given knowledge of the channel on which the media content isarriving, the media presentation device, the network server, and/oranother entity could then carry out one or more operations keyed to thechannel, such as determining and recording an extent to which mediacontent of that channel is being played, selectively replacing apredetermined portion of the media content with alternative content(such as a replacement advertisement), or superimposing channel-specificcontent over the media content for presentation along with the mediacontent, among numerous other possibilities.

One method to determine the channel on which media content is arrivingis to have the media presentation device (or perhaps an adjunct device)and/or a network server generate a digital fingerprint of the mediacontent that is being rendered by the media presentation device, and tothen compare that fingerprint with reference fingerprint dataestablished for media content known to be provided on particularchannels.

For example, a network server or other such entity may establish orotherwise have access to reference data that includes a referencefingerprint of a media stream being broadcast respectively on each ofvarious available channels and that maps each reference fingerprint tothe channel on which the represented media stream is being broadcast. Asthe media presentation device receives and renders media stream on agiven channel, the media presentation device may then generate afingerprint of that media stream and, through a network communicationinterface, report the generated fingerprint as a query fingerprint tothe network server for analysis.

The network server may then compare the reported query fingerprint withthe reference fingerprint data to find a match with a referencefingerprint and to thereby determine that the channel on which the mediacontent being rendered by the media presentation device is arriving isthe channel that the reference data maps with that referencefingerprint. Upon thereby determining the channel on which the mediacontent is arriving, the network server may then communicate anindication of the channel to the media presentation device, and themedia presentation device may take channel-specific action.Alternatively, the network server itself or another entity may takechannel-specific action based on the determined channel.

Unfortunately, however, a problem that can arise in this process is thatthe same media content may be broadcast on multiple different channels,whether concurrently or at different times, and so a query fingerprintof media content being rendered by a media presentation device may notcorrelate with just the channel on which the media content is arriving.For example, a sports game or political event might be broadcastconcurrently on multiple different channels, or a syndicated televisionor radio show might be broadcast on multiple different such channelsconcurrently or at different times. In these or other scenarios, if themedia presentation device is presenting such media content and generatesand provides a query fingerprint of the media content, that queryfingerprint could be matched with multiple reference fingerprints, andconsequently the channel-identification would be inconclusive.

As a specific example of this, consider a scenario where two differentvideo content providers both broadcast the same sports game, each on aseparate respective channel, and where an advertiser has a contract withjust one of the content providers to present a pop-up advertisement onthat provider's broadcast of the game. In this scenario, when a mediapresentation device is receiving and presenting one of these broadcasts,if the media presentation device generates and provides to the networkserver a query fingerprint of the broadcast, the network server maydetermine that the query fingerprint matches both content providers'broadcasts of the game, and so it would be unclear whether the mediapresentation device should present the pop-up advertisement or not.

Disclosed herein are methods and systems to help disambiguate channelidentification in a scenario where fingerprint data of media contentmatches multiple reference fingerprints corresponding respectively withmultiple different channels.

In accordance with the disclosure, a computing system will comparevarious reference fingerprints each representing a reference mediastream broadcast on a different respective known channel, and thecomputing system will determine that a plurality of the referencefingerprints match each other, thus defining a multi-match group of thematching reference fingerprints. Further, the computing system willdetermine that a query fingerprint representing a media stream beingrendered by a media presentation device matches the multi-match group,thus raising a question of which channel is carrying the media streamthat is being rendered by the media presentation device.

To resolve this question, according to the disclosure, the computingsystem will take into account one or more attributes of the queryfingerprint as a basis to determine which reference fingerprint of themulti-match group best matches the query fingerprint, thus establishingthat the channel on which the reference media stream represented by thedetermined reference fingerprint is broadcast is the channel on whichthe query media stream being rendered by the media presentation deviceis arriving.

In this process, the one or more attributes of the query fingerprintstream that the computing system considers as a basis to determine whichreference fingerprint of the multi-match group best matches the queryfingerprint could take various forms.

As an example, the one or more attributes could include a geographiclocation (e.g., a designated market area (DMA)) in which the mediapresentation device that provided the query fingerprint is located. Forinstance, if each of the reference fingerprints in the multi-match grouprepresents a reference media stream broadcast in a respective geographiclocation (e.g., a respective DMA), then the location of the mediapresentation device could help resolve the multi-match by identifying aproper subset of the members of the multi-match group having that sameassociated location.

And as another example, the one or more attributes could include anidentity of a media content distributor (e.g., cable-TV provider,satellite-TV provider, etc.) that provides media content for renderingby the media presentation device. For instance, if each of the referencefingerprints in the multi-match group represents a reference mediastream broadcast by a respective media content distributor, then theidentity of the media content distributor that provides media contentfor rendering by the media presentation device could help resolve themulti-match by identifying a proper subset of the members of themulti-match group having that same associated service provider. Otherexamples are possible as well.

Accordingly, in one respect, disclosed is a method for taking actionbased on channel identification, with multi-match detection based onreference stream comparison. In accordance with the method, a computingsystem obtains reference fingerprints that represent reference mediastreams broadcast on channels, where each reference fingerprintrepresents a respective one of the reference media streams, and whereeach reference media stream is broadcast on a respective channel. Andthe computing system detects, based on a comparison of the referencefingerprints with each other, that a plurality of the referencefingerprints match each other, the plurality of reference fingerprintsdefining a multi-match group.

Further, the computing system determines that a query fingerprintrepresenting a media stream being rendered by a media presentationdevice matches the multi-match group. And responsive to thatdetermination, the computing system then uses one or more attributes ofthe received query fingerprint as a basis to determine which referencefingerprint of the multi-match group best matches the query fingerprintand to determine that the channel on which the reference media streamrepresented by the determined reference fingerprint is broadcast is thechannel on which the media stream being rendered by the mediapresentation device is being broadcast, i.e., the channel carrying themedia stream being rendered by the media presentation device. Thecomputing system then takes action based on the determined channel, suchas causing supplemental channel-specific content to be presented by themedia presentation device for instance.

Additionally disclosed is a computing system configured to carry outoperations such as these. As disclosed, the computing system couldinclude a network communication interface and could include a controllercomprising a processing unit, data storage, and program instructionsstored in the data storage and executable by the processing unit tocarry out the operations. The computing system could include or be incommunication with a plurality of media-receiver devices configured toreceive the reference media streams on known channels and to provide thereference fingerprints representing those reference media streams. Thus,in practice, the computing system could compare the referencefingerprints generated by the media-receiver devices and thereby detectthe multi-match group. Further, the computing system could determinethat a query fingerprint representing a media stream being rendered by amedia presentation device matches the multi-match group, and thecomputing system could then consider one or more attributes of the queryfingerprint as a basis to disambiguate and thereby determine the channelon which the media stream being rendered by the media presentationdevice is being broadcast.

Still further, disclosed is a non-transitory computer readable mediumhaving stored thereon instructions executable by a processing unit tocarry out operations such as those noted above.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description, with reference where appropriate to theaccompanying drawings. Further, it should be understood that thedescriptions provided in this summary and below are intended toillustrate the invention by way of example only and not by way oflimitation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram of an example system in whichvarious disclosed principles can be applied.

FIG. 2 is a simplified block diagram of an example network arrangementin which a media presentation device communicates with a network serverto facilitate implementing various disclosed principles.

FIG. 3 is a flow chart depicting operations that can be carried out inaccordance with the present disclosure.

FIG. 4 is a simplified block diagram of an example network server.

FIG. 5 is a simplified block diagram of an example media presentationdevice.

DETAILED DESCRIPTION

Referring to the drawings, FIG. 1 is a simplified block diagram of anexample system in which various disclosed principles can be applied. Itwill be understood, however, that this and other arrangements andprocesses described herein can take various other forms. For instance,elements and operations can be re-ordered, distributed, replicated,combined, omitted, added, or otherwise modified. Further, it will beunderstood that functions described herein as being carried out by oneor more entities could be implemented by and/or on behalf of thoseentities, through hardware, firmware, and/or software, such as by one ormore processing units executing program instructions or the like.

As shown in FIG. 1, the example system includes one or more mediacontent sources 12 (e.g., broadcasters, web servers, etc.), one or moremedia content distributors 14 (e.g., multi-channel distributors, such ascable providers, satellite providers, over-the-air broadcast providers,web aggregators, etc.), one or more media content receivers 16 (e.g.,cable receivers, satellite receivers, over-the-air broadcast receivers,computers or other streaming media receivers, etc.), and one or moreclients or media presentation devices 18 (e.g., televisions or otherdisplay devices, loudspeakers or other audio output devices, etc.)

In practice, for instance, the media content sources 12 could benational broadcasters, such as ABC, NBC, CBS, FOX, HBO, and CNN, themedia content distributors 14 could be local affiliates and/or otherlocal content distributors in particular designated market areas (DMAs),and the receivers 16 and media presentation devices 18 could then besituated at customer premises, such as homes or business establishments.With this or other arrangements, the content sources 12 could delivermedia content to the content distributors 14 for distribution toreceivers 16 at customer premises, and the content distributors coulddistribute the media content to the receivers 16 on discrete channels(e.g., particular frequencies). Each receiver could then respond to userinput or one or more other triggers by tuning to a selected channel andoutputting to a media presentation device 18 the media content that isarriving on the selected channel. And the media presentation device 18could receive and render the media content (e.g., display or otherwisepresent the content).

In this arrangement, as the media presentation device receives andrenders this media content, the media presentation device may have noindication of the channel on which the media content is arriving, i.e.,of the channel to which the receiver is tuned. Rather, the mediapresentation device may be configured simply to receive the mediacontent as a media stream from the receiver and to render the receivedmedia content. Per the present disclosure, however, the mediapresentation device may be in communication with a network server andmay work with the network server to facilitate identification of thechannel and thus to facilitate taking useful channel-specific action.

FIG. 2 illustrates an example network arrangement in which a mediapresentation device 18 is in communication with a network server 20 viaa network 22, such as the Internet. In practice, the media presentationdevice 18 may sit as a node on a local area network (LAN) at customerpremises, with the media presentation device having an assigned InternetProtocol (IP) address on the LAN and the LAN having an IP address on theInternet. Further, the network server 20 may also be accessible at an IPaddress on the Internet. With this arrangement, the media presentationdevice may initiate and engage in IP communication with the networkserver via the Internet, to report a fingerprint of media content beingrendered by the media presentation device, so as to facilitate channelidentification and associated action.

As discussed above, the network server 20 or another entity operating inaccordance with the present disclosure could establish or have access toreference data 24 for media streams that are carried or scheduled to becarried on at least each of various channels accessible to the mediapresentation device 18. The reference data, which could be stored in arelational database or other form, could include reference fingerprintsthat represent media streams of various channels, with each referencefingerprint representing a respective reference media stream, and eachreference media stream being broadcast on a respective known channel.

In practice, for instance, some or all of the reference fingerprintscould represent reference media streams that are currently beingbroadcast on respective channels. For example, each such referencefingerprint could be a reference fingerprint stream representing a mostrecent period of time (e.g., on a sliding window basis) of a respectivereference media stream that is currently being broadcast on a respectivechannel. Alternatively or additionally, some or all of the referencefingerprints could be a reference fingerprint (e.g., referencefingerprint stream) of media content programs (e.g., televisionbroadcasts, radio broadcasts, streaming media files, etc.) definingmedia streams that are available or scheduled to be carried onparticular respective channels. As such, the reference data could mapeach reference fingerprint to a channel on which the associatedreference media stream (i.e., the media content uniquely identified bythe reference fingerprint) is being broadcast or is scheduled to bebroadcast.

In mapping reference fingerprints to channels, the reference data couldcharacterize the channels by various attributes, to help distinguish thechannels from each other. For example, where a receiver or other suchdevice provides multiple different channels that are user-selectable bychannel number, the reference data could characterize the channels bytheir respective channel numbers. And as another example, where eachchannel carries a particular content source's content, such as thecontent of one of a particular broadcaster for instance, the referencedata could characterize the channels by their respective contentsources' identities. Still further, where more than one contentdistributor (e.g., multichannel distributor) distribute content sources'content, the reference data could characterize the channels by theirrespective content distributors' identities. And for channels that arebroadcast in particular geographic locations, such as a particular DMAs,the reference data could characterize the channels by their broadcastlocations. In practice, the reference data could correlate eachreference fingerprint with one or more of these or other attributes.

The network server 20 or other entity operating in accordance with thepresent disclosure could establish or obtain some or all of thisreference data by analyzing media content arriving on each of thevarious channels (e.g., at least the media stream arriving respectivelyon each of various channels that are available to a receiver that servesthe media presentation device). To facilitate this, as shown, the servercould include or be interconnected with one or more media receiverdevices 26 that are configured to receive media streams from one or moreof the media content distributors 14 on various channels in much thesame way that receivers would be configured to receive content atcustomer premises. For instance, the server could include or beinterconnected with one or more cable-TV set top boxes, radios,computers, or other media receivers, or could be configured to emulateone or more such receivers. The server could then be configured toreceive and analyze the respective media stream arriving on each channeland to generate for each channel a respective reference fingerprint ofthe channel's media content, using any media fingerprinting process nowknown or later developed (e.g., computing a hash on a per frame or otherbasis, or otherwise identifying, extracting and digitally representingcomponent features unique to the media content).

The server could be configured to receive concurrently on multiple suchchannels (perhaps all of the channels) and to analyze and generaterespective fingerprints for the channels in parallel, or the servermight be configured to hop from channel to channel, possibly repeatedlycycling through the channels, to analyze and generate a respectivefingerprint for each channel. Further, the server could continue to dothis in real-time, saving a respective fingerprint of a most recent timewindow of media content for each channel, for reference. And the servercould record each channel's reference fingerprint in the reference datain association with a characterization of the channel (e.g., withattributes such as those noted above) and timestamp informationindicating a time of receipt of the associated media stream. Here, theserver would have knowledge of each channel (e.g., channel number), justas a receiver would normally have knowledge of the channel to which thereceiver is tuned. Further, the server may have access to guideinformation or other such data specifying attributes of each suchchannel (e.g., content source identity, content distributor identity,location, etc.), so that the server can determine and record channelattributes respectively for each reference fingerprint.

Alternatively or additionally, the server could receive such referencefingerprints from external media receiver devices 28, such as from mediareceiver devices located in various respective DMAs. This implementationcould be especially useful for channels that are provided only inspecific DMAs. In this implementation, each media receiver device 28,perhaps interworking with an associated computer server, could beconfigured to receive and analyze the respective media stream arrivingon each channel available to the media receiver device, to generate foreach such channel a reference fingerprint (e.g., reference fingerprintstream) of the channel's media stream using any fingerprint process nowknown or later developed, and to provide the reference fingerprint(e.g., periodically or continuously) via a network connection to theserver 20. Further, the media receiver device 28 could provide server 20with information characterizing the channel on which the media streamrepresented respectively by each such reference fingerprint is arriving,such as the channel number and other attributes noted above. And theserver could thus record each receive fingerprint along with itsassociated channel attributes.

Still alternatively or additionally, the server could also receive orperhaps establish reference fingerprints for media content programs thatare available or scheduled to be provided on particular channels. Forinstance, providers or distributors of various media content programscould generate reference fingerprints for the media content programs(defining media streams), again using any media fingerprinting processnow known or later developed, and those reference fingerprints could beprovided to the server. Alternatively, the server could receive advancecopies of the media content programs and could itself generate suchreference fingerprints. Further, the server could receive or determinefrom program guide information the channel on which the media contentprogram is available or scheduled to be provided, and perhaps a day andtime at which the media content is scheduled to be provided. The servercould then record each media content program's reference fingerprint inthe reference data in association with the channel on which the mediacontent program is available or scheduled to be carried, likewise withassociated channel attributes, and perhaps in association with a day andtime at which the media content program is scheduled to be provided.

Given this or other such reference data, when the server is presentedwith a query fingerprint of a media stream that was received on anunknown channel, the server could match the query fingerprint with oneof the stored reference fingerprints, using any fingerprint matchingprocess now known or later developed, and could thereby conclude thatthe media stream at issue arrived on the channel that the reference datamaps to the matching reference fingerprint. Thus, if the server is facedwith a query fingerprint of a media stream that is being rendered by themedia presentation device 18, the server could compare the queryfingerprint with the reference fingerprints in the reference data. Andif the server thereby finds a matching reference fingerprint, then theserver could identify the channel that the reference data maps with thematching reference fingerprint and could conclude that that is thechannel on which the media presentation device is receiving the mediastream (i.e., that that is the channel carrying the media content beingrendered by the media presentation device). In turn, the server couldresponsively take a channel-specific action based on the identifiedchannel, or cause or enable one or more other entities to take achannel-specific action based on the identified channel.

To facilitate this, the media presentation device 18 or another entitycould be configured to generate a query fingerprint of the media streamthat is being rendered by the media presentation device and to transmitthe query fingerprint to server 20 for analysis.

For instance, as shown in FIG. 2, the media presentation device couldinclude a fingerprint generator 30, which could be configured togenerate a digital fingerprint of the media stream being rendered by themedia presentation device. Such a fingerprint generator could beconfigured to generate the fingerprint of the media stream as the mediapresentation device is receiving the media stream on a channel from areceiver 16 and/or as the media presentation device is processing themedia stream for presentation. As such, the fingerprint generator couldreceive as input a copy of the media stream arriving at the mediapresentation device from the receiver and/or being processed forpresentation by the media presentation device, and to apply any mediafingerprinting process now known or later developed to generate afingerprint of the media stream.

In practice, the fingerprint generator could be configured to generatesuch a fingerprint as a fingerprint stream on an ongoing basis, such ason a per frame basis (e.g., per key-frame basis) or other basis. And themedia presentation device could be configured to transmit thefingerprint as a query fingerprint via network 22 to a server 20 foranalysis. By way of example, the media presentation device could beconfigured to periodically or from time to time transmit to the server aquery fingerprint representing a latest frame, series of frames or otherportion of the media stream being rendered by the media presentationdevice. In particular, the media presentation device could generate amessage carrying the latest generated query fingerprint, along with oneor more timestamps and/or other such data as well as an identifier ofthe media presentation device, and could transmit the message to theserver's IP address. And the server could thereby receive the queryfingerprint for analysis.

Alternatively, the media presentation device could transmit to theserver, and the server could thus receive, various data regarding themedia stream being rendered by the media presentation device, likewiseon an ongoing basis or other basis, to enable the server itself oranother entity to generate a fingerprint of the media stream beingrendered by the media presentation device. For example, the mediapresentation device could transmit to the server portions of the mediastream being rendered by the media presentation device, such asindividual frames (e.g., snapshots) or other segments of the mediastream. And the server could generate a fingerprint of that mediacontent for analysis.

Through the process described above, the server could then compare thefingerprint of the media stream being rendered by the media presentationdevice with the reference fingerprints in the reference data, using anydigital fingerprint comparison process now known or later developed. Andas noted above, if the server thereby finds a matching referencefingerprint, the server could determine the channel that the referencedata maps with the matching reference fingerprint and could concludethat the determined channel is the channel carrying the media streambeing rendered by the media presentation device.

In response to thereby determining the channel at issue, the servercould then take, or cause to be taken, one or more channel-specificactions based on the determination of the channel. In particular, theserver itself could take action based on the channel determination, orthe server could signal to another entity, perhaps to the mediapresentation device, to cause the other entity to take action based onthe channel determination.

For example, the server could record the fact that the mediapresentation device is presenting content of that particular channel, aspart of a channel rating or analytics system to measure the extent towhich particular channels are being presented by media presentationdevices. For instance, the media presentation device could regularly(e.g., periodically) report to the server a fingerprint of the mediastream that the media presentation device is rendering, and the servercould carry out processes such as those discussed herein to determinethe channel being rendered. Each time the server thus determines that achannel is being rendered, the server could add to a count or otherstatistic of the channel being rendered, as data to indicate the extentto which that channel is being presented.

As another example, the server could responsively cause the mediapresentation device to present supplemental content, such as a pop-upadvertisement as discussed above, a commercial break, or a channelidentification, among other possibilities, possibly as a replacement forone or more portions of the media stream being rendered by the mediapresentation device. For instance, given knowledge of the channel atissue, the server could generate or select (e.g., from server datastorage) particular supplemental media content associated specificallywith the determined channel and could transmit the supplemental mediacontent to the media presentation device for the media presentationdevice to present in conjunction with the media stream that the mediapresentation device is receiving from the receiver. The mediapresentation device could thus receive the supplemental media contentfrom the server and present it in conjunction with the media stream thatthe media presentation device is receiving from the receiver.

In practice, this process could involve the server receiving inreal-time from the media presentation device a fingerprint of the mediastream being rendered by the media presentation device and the serverdetermining that the received fingerprint matches a referencefingerprint of a media stream that is being broadcast, or is scheduledfor current broadcast, on a known channel.

Note that, in some cases, there may be a time difference between whenthe media presentation device renders media content and timestamps andtransmits the fingerprint to the server and when the server receives themedia content on the known channel and otherwise has the referencefingerprint timestamped. The server could account for this timedifference by comparing the received fingerprint over a sliding windowof the reference fingerprint or vice versa. Further, the server couldaccount for this time difference when taking action in response to adetermined match between the received fingerprint and the referencefingerprint. For example, if the media presentation device receives themedia content sufficiently earlier than the server's timestamp for thecontent (e.g., more than a few seconds earlier), the server could stillidentify a fingerprint match and could record analytics data. But inresponse to detecting that time difference, the server might forgohaving the media presentation device present associated supplementalcontent, to help avoid a situation where the media presentation devicepresents the supplemental content too late (e.g., is out of sync) from auser's perspective. On the other hand, if the server detects afingerprint match for a sufficient period of time and/or determines thatthe matching content will continue, the server may have the mediapresentation device present supplemental content even when faced withsuch a time difference.

In any event, through these or other such processes, a network server orother entity can determine the channel on which the media presentationdevice is receiving the media content at issue. And once the entitydetermines the channel, the entity may then take action based on thechannel determination. Alternatively, the entity could signal to anotherentity, perhaps back to the media presentation device, to cause theother entity to take action based on the channel determination. Otherexamples are possible as well.

In line with the discussion above, the server 20 or other entity thathas access to reference data as noted above could be configured toidentify multi-match scenarios, by detecting among the reference datavarious reference fingerprints that match each other.

In one implementation, for instance, the server could regularly analyzethe reference data in search of multi-match scenarios, comparing pairsof reference fingerprints in the reference data using any now known orlater developed media fingerprint matching process, in an effort to findreference fingerprints that match each other. Upon finding each suchmatch of at least two of the reference fingerprints, the server couldthen flag the reference fingerprints as being a multi-match group. Sucha flag could indicate that there will be a potential ambiguity if thefingerprint of media stream being rendered by the media presentationdevice matches any of the reference fingerprints in the flaggedmulti-match group. The server could flag the reference fingerprints asbeing a multi-match group in various ways. For instance, the servercould cross-reference the reference fingerprints of the multi-matchgroup in the reference data, to indicate that they are members of amulti-match group. Alternatively, the server could store a single copyof the reference fingerprint and associate that copy with variouschannels on which the media content represented by the referencefingerprint is carried.

With this implementation, when the server receives from a mediapresentation device a query fingerprint representing a media streambeing rendered by the media presentation device and the serverdetermines that the query fingerprint matches a reference fingerprint,the server could then readily determine from the reference data whethera multi-match situation exists. If the matching reference fingerprint isnot flagged or otherwise indicated as being a member of a multi-matchgroup, then the server could conclude that a single-match situation(rather than a multi-match situation) exists, in which case, asdiscussed above, the server could then readily determine from thereference data the channel associated with the matching referencefingerprint and could conclude that that is the channel carrying themedia stream being rendered by the media presentation device. Whereas,if the matching reference fingerprint is flagged as being a member of amulti-match group, then the server could conclude that a multi-matchsituation (rather than a single-match situation) exists, in which casethe server may need to perform a disambiguation process to help identifythe channel at issue from among those associated with the referencefingerprints of the multi-match group.

Note that the process of detecting a multi-match situation involving twoor more reference fingerprints could be performed even if the tworeference fingerprints represent the same media stream carried on twodifferent channels at a time delay relative to one another, i.e., wherethe presentation of the media content on one of the channels is timeshifted in relation to presentation of the same media content on anotherone of the channels. The fingerprint matching process could account forthis time shifting and could still find a match if the fingerprintsotherwise match each other, such as by comparing one fingerprint over asliding window with the other fingerprint. For instance, the process offinding matching reference fingerprints within the reference data couldinvolve searching for reference fingerprints that match one another andare presented and/or scheduled at respective times that are within athreshold time interval of each other.

By way of example, consider a scenario where largely the same mediastream is being presented on two channels for which the server or anexternal entity establishes respective reference fingerprintsrepresenting each channel's media stream, but where the media stream onone channel is slightly earlier in time than the media stream on theother channel. In that case, a first of the reference fingerprints couldbe considered an earlier or faster reference fingerprint representingthe first media stream, and the second reference fingerprint could beconsidered a later or slower reference fingerprint representing thesecond media stream. Notwithstanding the slight time shift between thesereference fingerprints, the server could still find that the referencefingerprints match each other, such as by comparing them over a slidingwindow or otherwise accounting for the slight time shift between them,and the server could flag or tag both of the reference fingerprints asbeing in a multi-match group with each other. Consequently, if theserver then receives a query fingerprint representing media contentbeing rendered by the media presentation device and determines that thequery fingerprint matches the earlier reference fingerprint, the servercould essentially predict that a multi-match situation exists, even ifthe server has not yet detected that the query fingerprint matches thelater reference fingerprint.

As noted above, when the server or other entity carrying out thisanalysis detects that the fingerprint of the media content beingrendered by the media presentation device matches multiple referencefingerprints corresponding with multiple channels, the entity couldapply a disambiguation process based on consideration of one or moreattributes of the query fingerprint, to determine which referencefingerprint of the multi-match group best matches the query fingerprint.Here, the one or more attributes of the query fingerprint could be anycontext information associated with the query fingerprint.

By way of example, as noted above, the one or more attributes couldinclude a geographic location (e.g., DMA) in which the mediapresentation device that provided the query fingerprint is locatedand/or an identity of the a media content distributor (e.g., cable-TVprovider, satellite-TV provider, etc.) that provides media content forrendering by the media presentation device. For instance, as notedabove, if each of the reference fingerprints in the multi-match grouprepresents a reference media stream broadcast in a respective geographiclocation (e.g., a respective DMA), then the location of the mediapresentation device could help resolve the multi-match by identifying aproper subset of the members of the multi-match group having that sameassociated location. And as further noted above, if each of thereference fingerprints in the multi-match group represents a referencemedia stream broadcast by a respective media content distributor, thenthe identity of the media content distributor that provides mediacontent for rendering by the media presentation device could helpresolve the multi-match by identifying a proper subset of the members ofthe multi-match group having that same associated service provider.

In practice, the server could determine these or other such attributesof the query fingerprint through communication with the mediapresentation device and/or through evaluation of the query fingerprint,as described by one or more the applications incorporated herein byreference. By way of example, when the media presentation devicetransmits the query fingerprint to the server, the media presentationdevice could provide the server with an indication of the mediapresentation device's location and/or associated media contentdistributor. As another example, the server could determine the mediapresentation device's location based on evaluation of the mediapresentation device's IP address. And as another example, the servercould determine that the query fingerprint contains one or morecomponents that are specific to a particular location and/or mediacontent distributor. Other examples are possible as well.

Through this process, the server could thus determine which referencefingerprint in the multi-match group best matches the query fingerprint,and the server could determine from the reference data the channelassociated with that matching reference fingerprint and could concludethat that is the channel carrying the media stream being rendered by themedia presentation device. And in turn, the server could then takechannel-specific action, such as causing the media presentation deviceto present supplemental channel-specific content, recording presentationof the identified channel, or the like.

Note that while the forgoing discussion focuses primarily on server 20identifying a multi-match situation and then disambiguating based onconsideration of context, some or all of the operations described couldalternatively be carried out by one or more other entities, in place ofor in cooperation with the server.

For instance, one or more of the operations could be carried out by themedia presentation device itself or by an adjunct system in localcommunication with the media presentation device. By way of example, themedia presentation device itself could be provisioned with or haveaccess to reference data like that described above, and the mediapresentation device could itself refer to the reference data to identifya multi-match situation, to disambiguate, and to identify the channelthat provides the media content being rendered by the media presentationdevice. Further, the media presentation device could then itself takechannel-specific action, such as presenting channel-specific content,recording channel presentation, or the like, or could cause one or moreother entities to take such action. Other variations are possible aswell.

FIG. 3 is next a flow chart depicting a method that can be carried outin line with the discussion above. One or more of the operations in themethod depicted by FIG. 3 could be carried out by a computing system,which could include a network server, a media presentation device,and/or one or more entities operating on behalf of or in cooperationwith these or other entities. Such a computing system could include aprogrammed processing unit or the like, configured to carry out one ormore of the method operations. Further, a non-transitory computerreadable medium (e.g., disc storage, flash storage, or the like) couldhave stored thereon instructions executable by a processing unit tocarry out the various depicted operations.

As shown in FIG. 3, at block 32, the method includes the computingsystem obtaining reference fingerprints that represent reference mediastreams (comprising video content and/or audio content), where eachreference fingerprint represents a respective one of the reference mediastreams, and where each reference media stream is broadcast on arespective channel. Further, at block 34, the method includes thecomputing system detecting, based on a comparison of the referencefingerprints with each other, that a plurality of the referencefingerprints match each other, the plurality of reference fingerprintsthus defining a multi-match group.

At block 36, the method further includes the computing systemdetermining that a query fingerprint representing a media stream(comprising video content and/or audio content) being rendered by amedia presentation device matches the multi-match group. For instance,the computing system could determine that the query fingerprint matchesa representative fingerprint of the multi-match group, or that the queryfingerprint matches a reference fingerprint that is a member of themulti-match group. And at block 38, the method includes, responsive tothe determining, the computing system using one or more attributes ofthe received query fingerprint as a basis to determine which referencefingerprint of the multi-match group best matches the query fingerprintand to determine that the channel on which the reference media streamrepresented by the determined reference fingerprint is broadcast is thechannel on which the media stream being rendered by the mediapresentation device is being broadcast.

At block 40, the method then includes the computing system taking actionbased on the determined channel. For example, the computing system couldcause supplemental channel-specific content to be presented by the mediapresentation device in conjunction with the media stream being renderedby the media presentation device, and/or the computing system couldrecord presentation of the determined channel for use in a channelratings system.

In line with the discussion above, in this process, the act of obtainingthe reference fingerprints could involve receiving the referencefingerprints from media receiver devices, with the media receiverdevices receiving the reference media streams on the channels andgenerating and providing the reference fingerprints representing thereceived reference media streams. Alternatively or additionally, the actof obtaining the reference fingerprints could include receiving thereference media streams and generating the reference fingerprints basedon the received reference media streams.

Further in line with the discussion above, the computing system could bean entity other than the media presentation device, in which case themedia presentation device could generate the query fingerprint and sendthe query fingerprint to the computing system, and the computing systemcould thus receive the query fingerprint from the media presentationdevice.

And additionally in line with the discussion above, the computing systemcould correlate each reference fingerprint with the channel on which thereference media stream represented by the reference fingerprint isbroadcast, to facilitate determining, based on the determined referencefingerprint, the channel on which the reference media stream representedby the determined reference fingerprint is broadcast. For instance, thecomputing system could record in data storage, in association with eachreference fingerprint, an indication of the channel on which thereference media stream represented by the reference fingerprint isbroadcast. Thus, when the computing system determines the referencefingerprint that best matches the query fingerprint, the computingsystem could determine the channel on which the associated referencemedia stream is broadcast, and the computing system could conclude thatthat is the channel on which the media stream being rendered by themedia presentation device is being broadcast.

FIG. 4 is next a simplified block diagram of an example system operablein accordance with the present disclosure. This system could represent anetwork server as described above, and/or one or more other entities(possibly including the media presentation device). As shown in FIG. 4,the example system includes a network communication interface 42, aprocessing unit 44, non-transitory data storage 46, any or all of whichcould be integrated together or, as shown, communicatively linkedtogether by a system bus, network, or other connection mechanism 48.

Network communication interface 42 could comprise one or more physicalnetwork connection mechanisms to facilitate communication on a networksuch as network 22 discussed above, and/or for engaging in direct ornetworked communication with one or more other local or remote entities.As such, the network communication interface could comprise a wirelessor wired Ethernet interface or other type of network interface, forengaging in IP communication and/or other type of network communication.

Processing unit 44, could then comprise one or more general purposeprocessors (e.g., microprocessors) and/or one or more specializedprocessors (e.g., application specific integrated circuits). Andnon-transitory data storage 46 could comprise one or more volatileand/or non-volatile storage components, such as optical, magnetic, orflash storage. As shown, non-transitory data storage 46 then storesprogram instructions 50, which could be executable by processing unit 44to carry out various operations described herein. For example, theprogram instructions could be executable to carry out the operationsdescribed above in connection with FIG. 3.

In practice, features of the present disclosure could also beimplemented in the form of a non-transitory computer readable mediumhaving stored thereon program instructions executable by a processingunit to carry out various operations. In particular, the operationscould include detecting, based on a comparison of reference fingerprintswith each other, that a plurality of the reference fingerprints matcheach other, the plurality of reference fingerprints defining amulti-match group, where each reference fingerprint represents arespective reference media stream being received on a respective knownchannel.

Further, the operations could include determining that a queryfingerprint representing a media stream being rendered by a mediapresentation device matches the multi-match group. And the operationscould include, responsive to the determining, using one or moreattributes of the query fingerprint as a basis to determine which onereference fingerprint of the multi-match group best matches the queryfingerprint and to determine that the channel on which the determinedreference stream represented by the determined reference fingerprint isbeing received is the channel carrying the media stream being renderedby the media presentation device. Further, the operations could theninclude taking action based on the determined channel. Various featuresdescribed above could be implemented in this context as well.

Finally, FIG. 5 is a simplified block diagram of an example mediapresentation device operable in accordance with the present disclosure.In line with the discussion above, this media presentation device couldtake various forms. For instance, it could be a television, computermonitor, or other device that operates to receive and render videocontent, and/or it could be a loudspeaker, a pair of headphones, orother device that operates to receive and render audio content. Numerousother examples are possible as well.

As shown in FIG. 5, the example media presentation device includes amedia input interface 52, a media presentation interface 54, a networkcommunication interface 56, a processing unit 58, and non-transitorydata storage 60, any or all of which could be integrated together or, asshown, communicatively linked together by a system bus, network, orother connection mechanism 62.

Media input interface 52 could comprise a physical communicationinterface for receiving media content to be presented by the mediapresentation device. As such, the media input interface could includeone or more wired and/or wireless interfaces for establishingcommunication with and receiving media content in analog or digital formfrom a receiver or other device or system. For example, the media inputinterface could include one or more interfaces compliant with protocolssuch as DVI, HDMI, VGA, USB, BLUETOOTH, WIFI, among numerous others.

Media presentation interface 54 could then comprise one or morecomponents to facilitate presentation of the received media content. Byway of example, the media presentation interface could comprise a userinterface such as a display screen and/or a loudspeaker, as well as oneor more drivers or other components for processing the received mediacontent to facilitate presentation of the content on the user interface.

Network communication interface 56 could comprise a physical networkconnection mechanism to facilitate communication on a network such asnetwork 22 discussed above, and/or for engaging in direct or networkedcommunication with one or more other local or remote entities. As such,the network communication interface could comprise a wireless or wiredEthernet interface or other type of network interface, for engaging inIP communication and/or other type of network communication.

Processing unit 58 could then comprise one or more general purposeprocessors (e.g., microprocessors) and/or one or more specializedprocessors (e.g., application specific integrated circuits). Andnon-transitory data storage 60 could comprise one or more volatileand/or non-volatile storage components, such as optical, magnetic, orflash storage. Further, as shown, data storage 60 stores programinstructions 64, which could be executable by processing unit 58 tocarry out various operations described here. For example, the programinstructions could be executable to generate on an ongoing basis afingerprint of a media stream being rendered by the media presentationdevice, based on analysis of the media stream being received at themedia input interface 52 and/or being processed at the mediapresentation interface 54, and to provide the generated fingerprint onan ongoing basis to facilitate channel identification and action asdescribed herein.

Exemplary embodiments have been described above. Those skilled in theart will understand, however, that changes and modifications may be madeto these embodiments without departing from the true scope and spirit ofthe invention.

What is claimed is:
 1. A method comprising: determining, based on acomparison of reference fingerprints with each other, that a pluralityof the reference fingerprints match each other, the plurality ofreference fingerprints defining a multi-match group, wherein eachreference fingerprint represents a respective reference media streamcarried by a respective known channel of media content, so that themulti-match group of the plurality of reference fingerprints correspondswith multiple channels of media content; upon determining that theplurality of reference fingerprints match each other, flagging at leasta given one of the reference fingerprints of the plurality as being amember of the multi-match group, wherein flagging the given referencefingerprint as being a member of the multi-match group comprisesassociating the given reference fingerprint with the multiple channels;detecting that a query fingerprint representing a media stream beingrendered by a media presentation device matches the given referencefingerprint; based on having detected that the query fingerprint matchesthe given reference fingerprint of the plurality, and based on havingflagged the given reference fingerprint as being a member of themulti-match group, determining that the query fingerprint matches eachreference fingerprint of the plurality; responsive to the determiningthat the query fingerprint matches each reference fingerprint of theplurality, using one or more attributes of the query fingerprint as abasis to identify a reference fingerprint of the plurality that bestmatches the query fingerprint; and determining, based on the identifiedreference fingerprint, what channel carries the media stream beingrendered by the media presentation device, wherein determining whatchannel carries the media stream being rendered by the mediapresentation device comprises determining, as the channel that carriesthe media stream being rendered by the media presentation device, thechannel on which the reference media stream represented by theidentified reference fingerprint is carried.
 2. The method of claim 1,further comprising taking action based on the determining what channelcarries the media stream being rendered by the media presentationdevice.
 3. The method of claim 2, wherein taking action comprisesreplacing a portion of the media content with alternative content. 4.The method of claim 3, wherein the alternative content comprises areplacement advertisement.
 5. The method of claim 1, further comprisingobtaining by the computing system the reference fingerprints, whereinobtaining the reference fingerprints comprises receiving the referencefingerprints from media-receiver devices, and wherein the media-receiverdevices receive the reference media streams on their respective channelsand generate and provide the reference fingerprints representing thereceived reference media streams.
 6. The method of claim 1, furthercomprising obtaining by the computing system the reference fingerprints,wherein obtaining the reference fingerprints comprises receiving thereference media streams and generating the reference fingerprints basedon the received reference media streams.
 7. The method of claim 1,wherein the computing system is an entity other than the mediapresentation device, and wherein the query fingerprint is generated bythe media presentation device, the method further comprising: receivingby the computing system, from the media presentation device, the queryfingerprint.
 8. The method of claim 1, wherein each reference mediastream and the media stream being rendered by the media presentationdevice comprises content selected from the group consisting of videocontent and audio content.
 9. A computing system comprising: a networkcommunication interface; a processing unit; non-transitory data storage;and program instructions stored in the data storage and executable bythe processing unit to carry out operations including: determining,based on a comparison of reference fingerprints with each other, that aplurality of the reference fingerprints match each other, the pluralityof reference fingerprints defining a multi-match group, wherein eachreference fingerprint represents a respective reference media streamcarried by a respective known channel of media content, so that themulti-match group of the plurality of reference fingerprints correspondswith multiple channels of media content, upon determining that theplurality of reference fingerprints match each other, flagging at leasta given one of the reference fingerprints of the plurality as being amember of the multi-match group, wherein flagging the given referencefingerprint as being a member of the multi-match group comprisesassociating the given reference fingerprint with the multiple channels,detecting that a query fingerprint representing a media stream beingrendered by a media presentation device matches the given referencefingerprint, based on having detected that the query fingerprint matchesthe given reference fingerprint of the plurality, and based on havingflagged the given reference fingerprint as being a member of themulti-match group, determining that the query fingerprint matches eachreference fingerprint of the plurality, responsive to the determiningthat the query fingerprint matches each reference fingerprint of theplurality, using one or more attributes of the query fingerprint as abasis to identify a reference fingerprint of the plurality that bestmatches the query fingerprint, and determining, based on the identifiedreference fingerprint, what channel carries the media stream beingrendered by the media presentation device, wherein determining whatchannel carries the media stream being rendered by the mediapresentation device comprises determining, as the channel that carriesthe media stream being rendered by the media presentation device, thechannel on which the reference media stream represented by theidentified reference fingerprint is carried.
 10. The computing system ofclaim 9, wherein the operations further comprise taking action based onthe determining of what channel carries the media stream being renderedby the media presentation device.
 11. The computing system of claim 10,wherein the action comprises causing a portion of the media content tobe replaced with alternative content.
 12. The computing system of claim11, wherein the alternative content comprises a replacementadvertisement.
 13. The computing system of claim 9, wherein theoperations further include obtaining the reference fingerprints, whereinobtaining the reference fingerprints comprises receiving the referencefingerprints via the network communication interface from media-receiverdevices, and wherein the media-receiver devices receive the referencemedia streams on their respective channels and generate and provide thereference fingerprints representing the received reference mediastreams.
 14. The computing system of claim 9, wherein the operationsfurther include obtaining the reference fingerprints, and whereinobtaining the reference fingerprints comprises receiving the referencemedia streams and generating the reference fingerprints based on thereceived reference media streams.
 15. The computing system of claim 9,wherein the computing system is an entity other than the mediapresentation device, wherein the query fingerprint is generated by themedia presentation device, and wherein the operations further include:receiving via the network communication interface, from the mediapresentation device, the query fingerprint.
 16. The computing system ofclaim 9, wherein the operations further include correlating eachreference fingerprint with the channel on which the reference mediastream represented by the reference fingerprint is carried, tofacilitate determining, based on the identified reference fingerprint,what channel carries the media stream being rendered by the mediapresentation device.
 17. A non-transitory computer readable mediumhaving stored thereon instructions executable by a processing unit tocarry out operations comprising: determining, based on a comparison ofreference fingerprints with each other, that a plurality of thereference fingerprints match each other, the plurality of referencefingerprints defining a multi-match group, wherein each referencefingerprint represents a respective reference media stream carried by arespective known channel of media content, so that the multi-match groupof the plurality of reference fingerprints corresponds with multiplechannels of media content; upon determining that the plurality ofreference fingerprints match each other, flagging at least a given oneof the reference fingerprints of the plurality as being a member of themulti-match group, wherein flagging the given reference fingerprint asbeing a member of the multi-match group comprises associating the givenreference fingerprint with the multiple channels; detecting that a queryfingerprint representing a media stream being rendered by a mediapresentation device matches the given reference fingerprint; based onhaving detected that the query fingerprint matches the given referencefingerprint of the plurality, and based on having flagged the givenreference fingerprint as being a member of the multi-match group,determining that the query fingerprint matches each referencefingerprint of the plurality; responsive to the determining that thequery fingerprint matches each reference fingerprint of the plurality,using one or more attributes of the query fingerprint as a basis toidentify a reference fingerprint of the plurality that best matches thequery fingerprint; and determining, based on the identified referencefingerprint, what channel carries the media stream being rendered by themedia presentation device, wherein determining what channel carries themedia stream being rendered by the media presentation device comprisesdetermining, as the channel that carries the media stream being renderedby the media presentation device, the channel on which the referencemedia stream represented by the identified reference fingerprint iscarried.
 18. The non-transitory computer readable medium of claim 17,wherein the operations further comprise generating the referencefingerprints and storing each reference fingerprint in correlation withthe channel on which the reference media stream represented by thereference fingerprint is carried.
 19. The non-transitory computerreadable medium of claim 17, wherein the operations further comprisetaking action based on the determining of what channel carries the mediastream being rendered by the media presentation device.
 20. Thenon-transitory computer readable medium of claim 17, wherein the actioncomprises causing a portion of the media content to be replaced withalternative content.